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Abstract 



The distribution of a certain IRT based person fit index to 
identify systematic cypes of aberrance is discussed. For the 
Rasch model it is proved that: (1) the joint distribution of 
subtest-residuals (the components of the index) is 
asymptotically multivariate normal, and (2) the distribution 
of the index is asymptotically chi-square. The parameters of 
these asymptotic distributions depend on whether ability of a 
person is known or estimated. Furthermore, the rate of 
convergence to the asymptotic distribution of the subtest— 
residuals is analyzed. In order to verify the results for 
short tests, a simulation study is conducted. 
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Asymptotic Distribution of an IRT Person Pit Index 

Introduction 

If an item response theory (IRT) model correctly 
describes the performance of a population of persons on a 
test., the item parameters are known and certain regularity 
conditions are satisfied, then the maximum likelihood (ML) 
estimators of ability are consistent, asymptotically normally 
distributed, and efficient (Hambleton & Swaminathan. 1985. 
chapters 5 and 7; Lord. 1983). Yet. these properties are no 
longer obvious in the presence of occasional or systematical 
deviations of the person's response behavior from the model, 
which are likely to arise in educational measurement and 
testing (Fyans. 1982). When occasional measurement 
disturbances, such as temporary carelessness/guessing are 
present, robust estimation of ability may be a good solution. 
Robust ability estimates show — at the cost of only a slightly 
increased estimation error for the regular patterns — a 
significantly decreased estimation error for the aberrant 
patterns (Jones. 1982; Mislevy & Bock. 1982; Wainer & Wright. 
1980). However, if systematic measurement disturbances, like 
unf amiliarity with the specific domain or copying/guessing to 
complete the test are expected, then it is more appropriate 
to detect and to diagnose those deviations rather than to 
correct them by robust estimation. 

For the detection of systematic aberrance, several 
person fit indices have been proposed (Levine & Drasgow. 
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1983: Smith. 1985; Trabin & Weiss. 1983). Unfortunately, the 
exact or asymptotic null distribution of these indices are 
not known and therefore their use is limited. 

In this paper, the asymptotic distribution of a simple 
modification of Smith's (1985) person fit index is 
investigated. For the case of known ability, the asymptotic 
distributions of subtest— residuals (the components of the 
index) as well as of the index itself are derived for a 
general IRT model. Next, still within the general framework 
of IRT, a basic system of equations connecting subtest- 
residuals both for known and ML estimated ability is found. 
Subsequently, the asymptotic distributions of the subtest- 
residuals and of the index when ability is estimated by the 
ML method are obtained, however, for the Rasch model only. 
Furthermore. for the subtest-residuals . the rate of 
convergence to the asymptotic distribution is evaluated. 

Person Fit Index for Diagnosing Aberrance 

Consider a test of n dichotomously scored items, and let 
Pi s P^O) = P(Xi=l|e) denote the probability of a correct 

response to item i (i = 1.2 n) for a person with ability 

9. As usual in IRT. it will be assumed that the P^'s are 
increasing functions of 9 . and that their first three 
derivatives with respect to 9 exist and are finite for all 
values of 9. Several examples of such functions, including 
the well-known one-, two- and three- paranieLer logistic ones. 

o 
u 
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can be found in Hambleton and Swaminathan (1985. chapter 3). 
A person's response pattern will be denoted by 
X s (X1.X2 . . . . .Xj^) . The responses are regarded as independent 
random variables given ability level 9. i.e., the local 
independence is assumed. Finally, it is assumed that all item 
parameters are known. 

The ML estimate of ability 9. 9. for a person with the 
response pattern X, is a solution of the likelihood equation 



(Hambleton & Swaminathan, 1985. chapter 5; Lord, 1983), where 
Pi' is the derivative of Pj^ with respect to 9, s 1-Pj^. 
and a hat above a function indicates that this function has 
to be calculated at 9. Lord (1983) found that for the three- 
parameter logistic model with known item parameters the ML 
estimate of ability is: 



(1) 




(2) 



- P 

consistent, i.e., 9 9 



as n 00 



and 



(3) 



asymptotically normally distributed and efficient, 
ft — ft J 

more precisely. — u ^ N(O.l) as n «; 

I'''^(9) 
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where 

(4) 1(9) e 

is the test information function, and ^ and ^ denote the 
convergence in probability and in distribution, respectively. 
In order to carry out the proof Lord needed the following 
assumptions: (1) 9 is bounded. (2) the item difficulty and 
discrimination parameters are bounded. (3) the pseudo- 
guessing paramet^ers are bounded away from one. and (4) the 
test is lengthened by adding strictly parallel forms, if 
person's response variables are non-identically distributed, 
then Lord's assumptions satisfy the regularity conditions of 
Bradley and Gart (1962) by which efficient ML estimation of 
ability is possible. In this paper we will use similar but 
more general assumptions, namely that there exist: a fixed 
ability interval [a.b] . fixed constants e1.e2.e3 > 0. and a 
fixed constant C > 0 such that 

(i) €1 < Pi(9) < l-€2 for all 9€{a.b]. 

and all i=l .2 n; 

and 

(ii) €3 < Pi' (9) < C for all 9e[a.b] 

and all i=1.2. . . . .n 
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Note that under assumptions (i) and (ii)r for each S^ta.b], 
(5) KG) oo as n -» 00. 

Furthermore, if (3) holds, then (2) is true if and only if 

(5) is satisfied. Hence, (5) is a necessary condition for (2) 
and (3) to hold together. Also, by arguments similar to those 
of Lord (1983), it can be shown that the ML estimators of 
ability, under assumptions (i) and (ii)r satisfy (2) and (3) 
for each 9e ta,b] . 

In this paper, the following between-subtests person fit 
index is used: 

(6) BF « ^ TT^—^ 

where 

Mj ^ HKEieSjXi) = SieSjPi' 

(7) 

^ Var(EieSjXi) = ^i^s^^iQi. 

and Sj (j = 1,2 J) indicates disjoint subtests obtained 

from a partition of the original test . The BF index is a 
multiplication of Smith's (1985) UB index by a constant: 
BF = (J-l)UB. Note that the BF value for a given pattern is 
the sum of the squared standardized residuals on the Sj 
subtests (in short, subtest-residuals) . Extreme (positive) 



Distribution of Person Index 

7 



values of the index indicate patterns with large deviations 
from the IRT expectations on the specific subtests. To detect 
such aberrant patterns Smith (1985. 1986) uses a posteriori 
fixed critical value justified by a simulation study (Smith. 
1988). 

Extreme values of the subtest-residuals show that 
aberrant behavior occurs on these subtests. Knowing the type 
of items in the subtests, attempts to diagnose the kind of 
aberrance may be made. Many person fit analyses are possible 
dependent on the manner of grouping items into sxibtests. For 
instance, items could be grouped into subtests according to: 
the increasing difficulty parameter of the items, their 
position in the i:est. the type of items, or the conceptual 
domains covered by the items . Each grouping may have a 
different diagnostic meaning. For example, a high positive 
value of the subtest-residual on the most difficult items may 
suggest/ that the person was guessing. Likewise, a high 
negative value of the subtest-residual on the item subset 
covering a specific domain may be an indication that the 
person has a poor knowledge of the domain. However, many 
person fit analyses should be conducted before a specific 
type of aberrance can be pointed out. 



Asymptotic Distribution of the Index 

Let u.s consider the asymptotic distribution of the 
subtest-residuals as well as of the index if the number of 
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items tends to infinity. We will investigate two separate 
cases: (1) the person's ability 9 is known, and (2) 9 is 
estimated by the ML method from his/her response pattern 

First, let us consider the case where ability is known. 
In this case, the subtest-residuals will be denoted by 



^ieSj^i *■ Mi 

(8) Yj s ^ ^, y H (Y1.Y2 Yj). 

^0 



Since Yj is the standardized sum of independent random 
variables {X^. i^^j} (by the assumption of local 
independence), we may apply the central limit theorem with 
the Liapunov condition (e.g. Rao, 1965, p. 107). So: 



(9) 



Mi s E(Xi) = Pi, ai2 = E(|Xi-Mi|2) = PiQi . 

Pi 3 E(IXi-Mil^) = PiQi(Pi2+Qi2) < P^Q^; 



and 



If the number of items in S-, , n^ . tends to infinity, then 
under assumption (i). the series Si^sj^i^i divergent and, 
consequently, the Liar anov condition lim^j-^ooLj = 0 is 
satisfied and Yj has asymptotically the standard normal 
distribution. Furthermore, the random variables Yj[ , Y2. . . . ,Yj. 
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being functions of disjoint subsets of the independent random 
variables X^'s. are independent, too. Hence, we can conclude 
that the random vec'^or (Y^ . Y2 . . . . . Yj) has asymptotically the 
non-singular standard normal distribution of rank J, i.e.. 

d 

(10) (Y1.Y2 Yj) -» N(O.I) as nj 00 (j=:1.2 J). 

where 0 is a vector of J zero's (the vector of the Y^'s 
means) and I is a JxJ identity matrix (the covariance matrix 
of the Y^ 's) . 

Now. let us investigate the asymptotic distribution of 

the BF index. Note that BF can be represented as a quadratic 

form in the variables Y]^.Y2. ■ ■ ..Yj. namely. BF = ^j^j^ ^ 

YIY' . where Y' is the transpose of Y. So. by (10) it becomes 
d 

clear that BF-»ZIZ'. where Z is N(O.I) distributed 
(Serf ling. 1980. p. 25. Corollary). Subsequently, we can 
conclude that BF is asymptotically chi— squared with J degrees 
of freedom, i.e.. 

(11) BF 4 x^j as n^ « (j=:l,2 J) 

(Serfling. 1980. p. 128. Lemma). 

Second, let us consider the asymptotic distribution of 
the subtest-residuals and of the index when the exact value 
of 9 is replaced by its ML estimate 0. In this case we will 
denote 



hi 
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(12) Y. s ^. Y s (Y1.Y2 Yj) 



and 



(13) BF s E-jYj2 = YIY' . 



where Mj and cr^ are calculated according to (7) at 9. Note 
that if the ML cibility estimates are used instead of true 
values, then due to (1), 9. and hence, the jjj's and o-^'s are 

functions of the X^'s. Therefore. ^1*^2 dependent 

random variables. The consequence of the dependency can 
easily be found for the Rasch model (RM) . For this model, 
where = l/(l+exp[-A(9-bi)] ) . (b^ is the difficulty of item 
i and A is a positive constant), we have P^' = PiQi* and 
from (4) for the subtest information function we obtain 

(14) 1^(9) H EieSjPi'^/PiQi = 2:igSjPiQi= '^j^- 
Putting this in (1) we thus have 

(15) Ei(Xi - Pi) = EjYjlj'^ = 0. 

Hence, for the RM. the random variables ^1*^2 ^n^st 

satisfy constraint (15). For J = 2. this means that and 
Y2 are perfectly correlated. The subtest-residual over all 
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items (if J = 1) is always equal to zero. In this way, 
similar results for other IRT models can be derived. Yet. as 
the dependency of Yi,Y2.....Yj results in a constraint we 
may expect that the joint distribution of the Y^'s is 
asymptotically a singular normal distribution of rank J-1 . 
and accordingly 3F is asymptotically chi-square distributed 
with J— 1 degrees of freedom. Let us verify these hypotheses. 

In order to find i,hc joint asymptotic distribution of 
(Y^ .Y2 . . . . . Yj) . the following system of equations derived 
from the def \iiitions of Yj and Yj can be used 



(16) = Y j - ^ ^ + (SieSjXi - Mj)(J^ - — ). 



where j = 1,2 J. Let us start with the question how 

the term (Mj - can be approximated. Note that 



(17) <rj2 = nj(PjQ^-s^2) 



where 



(18) Pj s (l/aj)SieSj^*i» ^ (l/nj)2i€Sj(^'i-Pj) . 

and s 1— P j . Since under assumptions (i) and (ii), (3) 
holds and from (4) I~^f9) = 0(n~^). we have 



Er|c J 6 



Distribution of Person Index 

12 



(19) 0 - e = OpCn--^) , 

where 0^ = OpCv^) denotes that for every €>0 there exists 
Kg>0 and Ng such that P(|On/Vjj|< Kg) > 1-e for all n>Ng , 
i.e., Ojj/Vn bounded in probablity. Now, applying the 
;ion to P^, and using (19) we obtain 

Pi - Pi = (6 -\Pi' + o^in-^), 

I P 

where Oj^ = ^r^^n^ denotes oJf/Vj^ -» 0. Summing both sides of 
the equation over ieSj, dividing them by cr-j , and applying 
(17) , we obtain 




Likewise, it can be shown that 



1 (Xi-Pi)Pi' 
-Si 



PiQi 



1 (Xi-Pi)Pi' (9-9) d (Xi-Pi)Pi' 
—Si + {Si ) 



PiQi 



(Tj d9 



PiQi 



(21) 



+ Op(l) 



Having calculated the derivative d{Ei(Xi-Pi)Pi' /PiQi)/d9 , it 
is then readily seen that the derivative may be written as 
[-1(9 )+SiKi(Xi-Pi)] , where the Ki's do not depend on Xi and 
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are bounded (again under assumptions (i) and (ii) and because 
all Pi'^ were assumed to be finite). Calculating the mean 
and variance of the last sum and using the Chebyshev 
inequality, we obtain E^Kj^CXj^-Pj^) = O^in^) . It is then easy 
to show, by the last result, (19) and (17) that 
[(e-G)/crj] (SiKi(Xi~Pi)] = op(l). Using this relation and (1) 
in (21 ) , we have 



9 - G 1 (Xi-Pi)Pi' 

(22) = Si + op(l) 

<rj I(G)crj PiQi 



Finally, applying the Chebyshev inequality. (20) and (17) in 
the similar wtiy. it can be shown that 



1 1 

(23) (EieSi^i - Mi)(- ) = op(l) 



Using the results (20). (22) and (23). the system of 
equations (16) can be replaced by the following cne 



SieSjPi' ^ (Xi-Pi)Pi' 

(24) = Y. ^ Si + Op(l). (j=l J) 

^ ^ KGXTj PiQi 



which will be the basis of our further considerations. 

Now. let us return to the RM where. Pi'= ^i^i (^^^ 
holds. For this model, the system of equations (24) may be 
expressed in a simple matrix notation. Denoting 
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R s ([ii(e)/i(e)]''*. ti2(e)/i(e)]''< tij(e)/i(e)]''4), 

and using (14) and an analog of the first equality in (15) at 
0, we obtain from (24) the following important c*atrix 
equation 

(25) Y' = Y'- R'RY' + z' = AY' + 2', 

where A s I-R'R, and z s (zi,Z2. . . rZj) is a vector of op(l) 
random variables which converges in probability to zero. Let 
us now assume that 

IjO) 

(iii) — ► ri as ni -» CO ( j=l , 2 , . . . , J) . 

lO) 

Then 

(26) R Rq H {T^'^. T2'^ rj^^), A Aq H I-Rq'Rq, 

and because of the identity EjljO) 1.- 1(9), ^j^j = 1 and 

(27) RqRq' = 1 

So, from the multivariate version of Slutsky's theorem (Rao,, 
1965, p. 102, Corollary (x)), we can conclude frorr (25) and 
(26) that 

J u 
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" d 

(28) y -> AoY' . 

- d 

Subsequent ely, by (10) we have Y' -> N(O.AolAo'), (Serfling, 

1980. p. 26. Application A). But AqIAq' = I-Rq'Rq, because Aq 

= Aq' and (27) holds. This means also that the matrix I— Rq'Rq 

is idempotent. and thus of rank r=rank (I-Rq'Rq) = trace (I— 

^o'^o^ = " turn as known,, the characteristic 

function of a N(0»I-Ro'Ro^ distributed variable Z is (t)(t) = 
-14t(I-Ro'Ro)t' 

e . Hence . by an orthogonal transformation 

T'=Ct' such that Tj = E-jt^r^''^, we can obtain Q(t) = 
Q(T) = Ej"]; Tj2 (Rao. 1965, chapter 3b3). This means that the 
total mass of Z is situated in the hyperplane ZjZjT^^^=0, So 
we may regard the following theorem as proved: 



Theorem . If for the Rasch model assumptions (i). (ii) and 
(iii) are satisfied and ability is estimated by the maximum 
likelihood method, then the joint distribution of the 
subtest-residuals is asymptotically a singular normal 
distribution of rank J-1, i.e.. 

(29) (Yi,Y2' Yj) ^ N(O.I-Ro'Ro) 

as nj -> oo ( j=i ,2, ... .J) . 

the total mass of which is situated in the hyperplane 
From the theorem it immediately follows that the asymptotic 
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variance and correlation of the Yj subtest— residuals are 

(30) Var(Y.) -> 1-r. , Corr(Y. . Y, ) -> ^ ^ 

^ ^ ^ ^ {(l-rj)(l-rjc)}^ 

as n^, njc -> oo. If J = 2. then Corr(Y^ ,Yj^) -> -1 . This 

corresponds to the result on page 10, where the correlation 

has been shown to be perfect. Considering also that SjZjrj^'^=0 

corresponds to (15), the asymptotic distribution of the Y-j's 

possess the same two properties as their exact distribution. 

Having the limiting distribution of the subtest- 

residuals, the limiting distribution of BF can be considered. 

- d 

By (13) and (28) we have BF -> YAo'lAoY' (Serfling. 1980. 
p. 25, Corollary), where A^'IAq = I-Rq'r^ is, as was 
mentioned, idempotent and of rank J— 1 . Applying the Fisher- 
Cochran theorem (Serf ling, 1980, p . 128 , Lemma) , we may 
conclude: 

Corollary . If for the Rasch model assumptions (i), (ii; and 
(iii) are satisfied and ability is estimated by the maximum 
likelihood method, then the index is asymptotically chi- 
square distributed with J-1 degrees of freedom., i.e., 

- d ^ 

(31) BF -> Xj^i as nj ^ 00 (j=1.2, J). 
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Results (10). (11). (29) and (31) may be v^ry useful in 
applicati<./n3 . provided that the error of the approximations 
for tests of realistic length is not too large. 

Let us restrict ourselves to the error of the standard 
normal approximation for the subtest— residuals when ability 
is known (see (10)). For any independent random variables 
Xi's that have finite third absolute moments, the following 
inequality holds 



where Fj(x) is the distribution of the Rj standardized 
summands. §(x) is the standard normal distribution, and C is 
an universal constant independent of n^ and of any 
characteristic of the X^'s (Se fling. 1980. p. 33). It is p^sy 
to show that if our specific X^'s are identically distributed 
(Pi = P and thus 0^=0 for all ieSj). then the order of Dj . 
which is given by the inequality, cannot be improved. From 
the classical theorem of de Moivre-Laplace it follows that 
the function Fj(x) (here the distribution of Yj) is 
discontinuous at the points X3^=(k-njP)/(njPQ)^'^. 
(k=0. 1 . . . . nj ) , with jumps asymptotically equal to 
{l/(2TTnjPQ)'^}exp(-X3^2/2) , Hence Dj is of order Oinj"'^) . On 
the other hand, using (9). we may conclude that Bj is of the 
same order. Thus,, in general, for the evaluation of the error 
it is sufficient to consider the B-^ bound in (32). 



(32) s sup|Fj(x)-$(x)| < C 



2ieSjE(IXi-Hi|3) 




^2i€SjE(IXi-^ii|2)}3/2 
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Substituting in (32) the moments of X^, being calculated 
in (9), applying Pi2+Q^2 - i-2PiQi and the Cauchy-Schwarz 

inequality (l/Hj ) (EigSj^i^i^^ - ^ieSj^^i^i)^ • ^^^n using 

(17) we obtain 

C 

(33) Dj < Bj < — [l-2(PjQj-.Sj2)] . 

Again with Pi=P for all ieSj , it is easy to show that the 
second inequality cannot be improved. So, the bound for Dj 
error of the standard normal approximation for Y-^ in (10), is 
a function of n^ as well as of the variability of the subtest 
Pi(0)'s at a fixed 9, Pj and Sj2. xhe inequalities show that 
the bound for D j , B j , mainly depends on the behavior of 
C/{nj(PjQj-Sj2)}!A Therefore, for a fixed Pj and Sj^. Bj 
is of order 0(nj"'*) . Next, for a fixed n j , the following 
conclusions can be drawn: (a) Bj is minimal when at 9 all 
Pi(9) = 1/2 (note that in this case Dj<C/(0 .25nj )''^) ; (b) if 
the average of the Pi(9)'s at 9, P j , tends to 0 or 1 , then Bj 
tends to infinity; (c) at a fixed average P j , the larger the 
variation of the Pi(9)'s, Sj2, the larger B j . These 
conclusions hold for an arbitrary IRT model for which (10) is 
satisf j ed. 

For the RM in particular, using (14) and (17) we obtain 

(34) Ij(9) = nj(PjQj-Sj2), 
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and from (33) 



C 



(35) 




2Ij/nj). 



The behavior of the subtest information Ij is illustrated in 
Figure 1(a) for two cases of Sj subtest: (1) of items with 
similar difficulties. and (2) of items with distant 
difficulties. However, in both cases the means of the 
difficulties are equal, i.e.. ^j(l) = ^j(2) ^o- Then for 
ability 9 very close to bo (where Pjal4), ^j(i) > ^j(2) 
because, as can be concluded from the Pj^(8)'s for the RM. 
Pj(l) « ^j(2) while < Sj^(2)- more extreme 

^j(l) < ^.1(2) because Pj^) < P-j(2) while s^^^^^^^ 
Sj^(2)- Of course, for 9 -» + Ij(l) and Ij(2) tend to zero 
irrespective of the difficulties of the items. If Pi(9) = 
P(9) for all ieSj . then bi=b and the highest possible 
information is obtained at 9 = b which is equal to 0.25nj. 
According to the behavior of Ij and (35). the bound for Dj 
error in the two cases is illustrated in Figure Kb). The 
minimal possible error is obtained at 9=b when Pi(9) = P(9) 
for all ieSj. However, for the subtest of Items with more 
spreaded difficulties, the standard normal approximation for 
the subtest-residual is better in a broader range of ability. 
With increasing the number of items in the subtest, the 
approximation becomes better and it is of order n"-^. 
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Numerical Results 

In order to examine the degree of approximation by the 
asymptotic distributions, a hypothetical test was designed 
according to the RM. The test was composed of 40 items with 
the item difficulties b^ sampled from the normal distribution 
with mean 0.00 and variance 1.562, investigate the effect 
of different grouping of items into subtests, all items were 
ordered and numbered corresponding to the values of (from 
the lowest to the highest) and then divided into a few 
subtests in three analyses. In the first analysis, two 
subtests of 20 items were formed (the first subtest consisted 
items with numbers from 1 to 20; the second subtest consisted 
items 21 to 40). In the second analysis, four subtests of 10 
items (items 1 to 10, items 11 to 20, etc.) and in the third 
analysis eight subtests of 5 items (items 1 to 5, items 6 to 
10. etc.) were constructed. The parameters P ^ , Sj and Ij of 
these subtests, calculated using (18) and (34) for ability 
9 =0.00, are presented in Table 1. 



Insert Table 1 about here 



Subsequently, five hundred patterns were generated for 

9 =0.00 according to the RM and the values of the subtest- 

residuals and of the index were calculated in the three 
analyses, as well. 
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The mean and variance of the empirical distribution of 
the subtost-residuals are given in columns 2 and 3 of Tables 
2. 3 and 4, and the mean and variance of the empirical 



Insert Tables 2 , 3 & 4 about here 



distribution of the index in columns 6 and 7, both for the 
case of known ability (i.e., 9 = 0.00) and the case of 
ability estimated by the ML method (applying the PML progra'n, 
see Gustafsson, 1981). The corresponding results for the 
asymptotic distributions, calculated using (10) and (11) for 
known ability, and using (29), (30) and (31) for the ML 
estimated ability, are given within brackets immediately 
below the empirical ones. As is easily seen, a significant 
agreement between the empirical and asymptotic means and 
variances was obtained. 

Furthermore, in order to investigate whether the 
empirical distributions can be approximated by their 
asymptotic ones , the JColmogorow-Smirnow and Chi-Square tests 
of goodness-of-f it were applied (with the help of the NEAR 
TESTS of the SPSS, see Hull and Nie, 1981). 

The results of the Kolmogorow-Smirnow test for the 
distribution of subtest-residuals are given in columns 4 and 
5 of Tables 2, 3 and 4. Here the theoretical distribution of 
the Kolmogorow-Smixnow test was assumed to be normal with 
mean and variance given within the brackets (i.e., the 
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appropriate asymptotic distribution). In columns 4 there are 
listed values of Dj defined in (32). For the case of known 
ability, values of Dj are found to be proportional to the 
reciprocal of the root of Ij (see (35) and Table 1). In 
addit.lon, th» results confirm the rules (a), (b) and (c). 
specified on page 18. However, it seems that the same 
conclusions hold for the case of the ML estimated ability. 
Columns 5 of Tables 2. 3 and 4 show that the probability of 
exceedance for the Kolmogorow— Smirnow statistic (and thus 
also for the Dj statistic) will be higher than the observed 
one. The results obtained for known ability indicate that the 
distribution of the subtest-residuals is not satisfactory 
approximated by the standard normal distribution, at least in 
our example of 20, 10 and 5 item subtests. However, the 
results obtained for ML estimated ability seem to be more 
promising. For instance, in Table 3 two of the four 
distributions of Yj can be regarded as normal at the 0.05 
significance level . This discrepancy may be due to the 
fact that the number of possible values of the Yj statistic 
is higher than that one of the Yj statistic. Therefore, for 
estimated ability the empirical distribution is smoother and 
with lower values of Dj . 

Finally, the results of the usual Chi— Square goodness- 
of— fit test for the distribution of the index are shown in 
the last three columns of Tables 2, 3 and 4. Here the data 
were pooled according to the 30%-, 50%-, 70%, 90%- and 95%-th 
percentiles of the appropriate chi— squared distribution (with 
the number of degrees of freedom according to (11) for the 
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case of known ability, and according to (31) for ML estimated 
ability). Using these percentiles, empty classes could be 
avoided. As can be seen, the null hypotht "ir of a chi-square 
distribution of the index cannot be rejected at the 0 . 05 
significance level. In summary, the asymptotic distributions 
may certainly be useful for the purpose of person fit 
analyses, even for the tests of relatively low length. 

Discussion 

IRT offers several person fit indices to identify 
systematic types of aberrance in a person's response 
behavior. Unfortunately, the proper use of these indices has 
so far been limited by an insufficient knowledge of their 
exact or asymptotic null distribution. Therefore. the 
asymptotic distribution of the person fit index and of the 
subtest-residuals given in lis paper should have some 
practical value., at least for the Rasch model and when 
ability is estimated by the ML method. So, for instance, in 
order to detect the type of eiberrance called un-. or super- 
familiarity with specific domains, the following procedure 
can be applied. First, the subsets of items covering the J 
domains concerned should be specified. Second, for a given 
pattern, the ML ability estimate 9 and the values of and 
BF must be calculated. If the value of BF is higher than the 
100*a% percentile of the distribution with J-1 degrees of 
freedom (see Corollary), then the pattern is classified as 
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aberrant with the Type I error equal to a . Third, in order 
to examine whether the detected aberrance is due to a 
particular domain, values of divided by its standard 

deviations (l—Ij/I)''^, for each of J subsets (see (30) and 
(iii)), have to be calculated. If one or more of these values 
are lower than the 100* (a/2 )% percentile or higher than the 
100*(l-a/2)% percentile of the standard normal distribution 
( see Theorem) , then we may conclude an interaction between 
the subtest scores and the domains. So, we may suspect that a 
person is un-familiar with one domain, while he/she is super- 
familiar with another. Of course, many other types of 
aberrance should be tested before we definitively conclude to 
the type of aberrance involved in the test— taking behavior of 
the given person. 

In order to determine the asymptotic distributions, 
assumptions (i), (ii) and (iii) have been made. These 
assumptions are rather technical than restrictive, thus, they 
can be easy realized when applying the Rasch model , 
Considering the fact that tests are usually designed for a 
given population of persons, they seem to be mild enough to 
satisfy all standard test settings. The first assumption 
stipulates that there must exist an interval of ability for 
which there are no items with the item characteristic curves 
(ICC's) closely approaching the zero and one asymptotes. As 
those items are just the ones which are too easy or too 
difficult to be included in the test for a given population, 
this assumption should easily be satisfied in practice. The 
second assumption requires that for the same ability interval 
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there are no items of which ICC's are neither almost 
horizontal nor almost vertical. This assumption is also met 
in most standardized test settings as items of which ICC's 
are very flat are considered to be too little discriminating 
to be adopted in the test. Likewise, items of which the ICC's 
are nearly vertical simply do not exist in practice. The last 
assumption stipulates that , with increasing the number of 
items in the test, the ratio's of subtest and test 
information approach certain constants. In practice, this is 
easy to realize as usually in the construction of a test 
items are sampled from subdomains in accordance with a fixed 
distribution. 
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Table 1 

Parameters of subtests for known ability (9 = 0.00) 

Subtest Number Subtest Parameter 
of items — = 



First Analysis 

1 20 0.701 2.109 3.768 

2 20 0.189 1.211 2.823 



Second Analysis 

1 10 0.825 0.467 1.397 

2 10 0.576 0.669 2.371 

3 10 0.282 0.556 1.968 

4 10 0.096 0.143 0.855 



Third Analysis 

1 5 0.882 0.201 0.510 

2 5 0.768 0.083 0.887 

3 5 0.646 0.385 1.124 

4 5 0.509 0.049 1.247 

5 5 0.340 0.373 1.104 

6 5 0.223 0.050 0.864 

7 5 0 126 0.089 0.546 

8 5 0.066 O.C'O 0.309 
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Table 2 

Mean and variance of empirical and asymptotic distributions, 
and results of goodness-of-f it tests from first analysis 



Sub Subtest- Index 

test Residual K-S Test Chi-Square Test 

Mean Var. P(Dn^>Do) Mean Var. PCX^^X^o ) 



Known Ability (G = 0.00) 

1 -0.03 0.96 0.127 0.000 

(0.00)(1.00) 1.90 3.60 7.617 2 0.179 

2 0.03 0.94 0.136 0.000 (2.00) (4.00) 
(0.00)(1.00) 



ML Estimated Ability (0=0) 

1 -0.02 0.42 0.073 0.010 

(0.00)(0.43) 0.97 1.86 8.727 1 0.120 

2 0.02 0.55 0.061 0.050 (1.00) (2.00) 
(0.00)(0.57) 



Note. The parameters of the asymptotic distributions are 
given within brackets. 



ERIC 



35 



Distribution of Person Index 

31 



Table 3 

Mean and variance of empirical and asymptotic distributions, 
and results of goodness of fit tests from second analysis 



Sub 


Subtest— 


Index 




test Residual 


K-S Test 


Chi-Square Test 




Mean 


Var. 


Do P(Dn''^>Do) Mean Var . 


X^^ df P(x2>x2^) 








Known Ability (0 = 0.00) 




1 


0.03 0 


.97 


0.177 0.000 






(0.00)(1 


.00) 






2 


-0.06 1 


.00 


0.144 0.000 






(0.00)(1 


.00) 


3.87 7.32 5 


.637 4 0.343 


3 


0.02 0 


94 


0.160 0.000 (4.00) (8.00) 






(0.00)(1 


00) 






4 


0.02 0 


97 


0.243 0.000 






(0.00)(1 


00) 












ML Estimated Ability (0=9) 




1 


0.07 0 


81 


0.080 0.003 






(0.01)(0 


79) 






2 


~0 . 03 0 


62 


0.058 0.070 






(0.00)(0 


64) 


2.95 5.65 10 


187 3 0.070 


3 


-0.01 0. 


69 


0.046 0.248 (3.00) (6.00) 






(0.00)(0. 


70) 






4 


-0.03 0. 


83 


0.119 0.000 






(0.00)(0. 


87) 







Note . The parameters of the asymptotic distributions are 
given within brackets. 
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Table 4 

Mean and variance of empirical and asymptotic distributions, 
and results of goodness of fit tests from third analysis 



Sub Subtest— 








Index 






test Residual 






Test 


Chi- 


-Square Test 


Mean Var . 


Do P(Dn^>Do) Mean Var. 


o 

Z o 


df P(z2>x2 ) 
o 








Known Ability (G = 0.00) 






1 0.05 0 


99 


0 


358 


0 


000 






(0.00)(1 


.00) 














2 -0.01 0 


98 


0 


238 


0 


.000 






(0.00)(1 


00) 














3 -0.03 1 


02 


0 


180 


0 


000 






(0.00)(1 


00) 














4 -0.05 1 


05 


0 


197 


0 


000 






(0.00)(1 


00) 








8,05 18.85 


8.737 


8 0.120 


5 0.07 1 


00 


0 


193 


0 


000 (8,00)(16.00) 






(0.00)(1 


00) 














6 -0.04 1 


02 


0 


239 


0 


000 






(0.00)(1 


00) 














7 -0.01 0 


.96 


0. 


Cll 


0 


000 






(0.00)(1 


00) 














8 0.05 1 


02 


0. 


403 


0 


000 






(0.00)(1 


00) 























ML Estimated Ability (8=9) 


1 


0 


.09 0 


.89 


0.277 0.000 




(0 


.00)(0 


.92) 




2 


0 


.03 0 


.89 


0.083 0.002 




(0 


.00)(0 


.87) 




3 


-0 


.00 0 


86 


0.059 0.062 




(0 


00)(0 


83) 




4 


-0 


04 0 


87 


0.066 0.027 




(0 


00)(0 


81) 


7.13 16.65 2.727 7 0.742 


5 


0 


05 0 


90 


0.063 0.039 (7.00)(14.00) 




(0 


00)(0 


83) 


6 


-0 


07 0 


89 


0.135 0.000 




(0 


00)(0 


87) 




7 


-0 


04 0 


92 


0.230 0.000 




(0. 


00)(0. 


92) 




8 


0. 


01 0. 


89 


0.330 0.000 




(0. 


00)(0. 


95) 





Note . The parameters of the asymptotic distributions are 
given within brackets . 
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(b) Bound for Error of Standard Normal Approximation 




Figure 1 . Information function and bound for error of 
standard normal approximation for subtest— 
residuals . in two cases of subtest : ( 1) of items 
with similar difficulties (dashed line), and (2) of 
items with distant difficulties (solid line). 
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